Skip to content

Conversation

@justadogistaken
Copy link

Motivation

ref #14478

It will be unexpected exit when eagle & grammar are enabled at the same time. It dues to that reasonerGrammar is not terminated correctly.

[2025-12-04 22:10:40] Scheduler hit an exception: Traceback (most recent call last):
  File "/sglang/python/sglang/srt/managers/scheduler.py", line 2679, in run_scheduler_process
    scheduler.event_loop_normal()
  File "/spec/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/sglang/python/sglang/srt/managers/scheduler.py", line 980, in event_loop_normal
    result = self.run_batch(batch)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/sglang/python/sglang/srt/managers/scheduler.py", line 2034, in run_batch
    batch_result = self.model_worker.forward_batch_generation(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sglang/python/sglang/srt/speculative/eagle_worker.py", line 295, in forward_batch_generation
    self.verify(batch, spec_info)
  File "/sglang/python/sglang/srt/speculative/eagle_worker.py", line 701, in verify
    vocab_mask = generate_token_bitmask(
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/sglang/python/sglang/srt/speculative/spec_utils.py", line 654, in generate_token_bitmask
    traverse_tree(
  File "/sglang/python/sglang/srt/speculative/spec_utils.py", line 620, in traverse_tree
    dfs(0, retrieve_next_token, retrieve_next_sibling, -1)
  File "/sglang/python/sglang/srt/speculative/spec_utils.py", line 600, in dfs
    dfs(
  File "/sglang/python/sglang/srt/speculative/spec_utils.py", line 597, in dfs
    grammar.fill_vocab_mask(allocate_token_bitmask, curr)
  File "/sglang/python/sglang/srt/constrained/reasoner_grammar_backend.py", line 69, in fill_vocab_mask
    self.grammar.fill_vocab_mask(vocab_mask, idx)
  File "/sglang/python/sglang/srt/constrained/xgrammar_backend.py", line 100, in fill_vocab_mask
    self.matcher.fill_next_token_bitmask(vocab_mask, idx)
  File "/spec/lib/python3.11/site-packages/xgrammar/matcher.py", line 308, in fill_next_token_bitmask
    return self._handle.fill_next_token_bitmask(bitmask, index, debug_print)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: [22:10:40] /project/cpp/grammar_matcher.cc:556: Check failed: (!IsStopTokenAccepted()) is false: GrammarMatcher has terminated after accepting the stop token, but is trying to find the next token mask

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

@gemini-code-assist
Copy link
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@LorrinWWW
Copy link
Collaborator

Thanks! There is actually a duplicated PR #14464 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants